Modular SRV Reinforcement Learning Architectures for Non-linear Control
نویسندگان
چکیده
This paper demonstrates the advantages of using a hybrid reinforcement–modular neural network architecture for non-linear control. Specifically, the method of ACTION-CRITIC reinforcement learning, modular neural networks, competitive learning and stochastic updating are combined. This provides an architecture able to both support temporal difference learning and probabilistic partitioning of the input space. The latter is formed with the aid of competitive learning algorithms, so as to ensure suitable partitioning of the experts in the modular network. Application of this methodology to the pole-balancing benchmark non-linear control problem demonstrates superior partitioning of the input space, bettering that of equivalent reinforcement networks; whilst avoiding the learning-to-learn nothing effect, as is often the case when performing gradient decent over problems requiring adaptation over long temporal dependencies.
منابع مشابه
Inter-module credit assignment in modular reinforcement learning
Critical issues in modular or hierarchical reinforcement learning (RL) are (i) how to decompose a task into sub-tasks, (ii) how to achieve independence of learning of sub-tasks, and (iii) how to assure optimality of the composite policy for the entire task. The second and last requirements are often under trade-off. We propose a method for propagating the reward for the entire task achievement ...
متن کاملDeep Reinforcement Learning for Robotic Manipulation - The state of the art
The focus of this work is to enumerate the various approaches and algorithms that center around application of reinforcement learning in robotic manipulation tasks. Earlier methods utilized specialized policy representations and human demonstrations to constrict the policy. Such methods worked well with continuous state and policy space of robots but failed to come up with generalized policies....
متن کاملAssociative Reinforcement Learning of Real-valued Functions
|Associative reinforcement learning (ARL) tasks de ned originally by Barto and Anandan [1] combine elements of problems involving optimization under uncertainty, studied by learning automata theorists, and supervised learning pattern-classi cation. The stochastic real-valued (SRV) unit algorithm [6] has been designed for an extended version of ARL tasks wherein the learning system's outputs can...
متن کاملReusable Reinforcement Learning for Modular Self Motivated Agents
Presented topic is from the research fields called Artificial Life and Artificial Intelligence (AI). In this paper, there is presented novel approach to designing agent architectures with its requirements. The approach is inspired by inherited modularity of biological brains and agent architectures are represented here as set of given reusable modules connected into a particular topology. This ...
متن کاملModular Neural Networks and Reinforcement Learning
We investigate the effect of modular architecture in an artificial neural network for a reinforcement learning problem. Using the supervised backpropagation algorithm to solve a two-task problem, the network performance can be increased by using networks with modular structures. However, using a modular architecture to solve a two-task reinforcement learning problem will not increase the perfor...
متن کامل